Learning reward functions from diverse sources of human feedback: Optimally integrating demonstrations and preferences

Reward functions are a common way to specify the objective of robot. As designing reward can be extremely challenging, more promising approach is directly learn from human teachers. Importantly, data teachers collected either passively or actively in variety forms: passive sources include demonstrations (e.g., kinesthetic guidance), whereas preferences comparative rankings) elicited. Prior research has independently applied learning these different sources. However, there exist many domains where multiple complementary and expressive. Motivated by this general problem, we present framework integrate information, which users. In particular, an algorithm that first utilizes user initialize belief about function, then probes with preference queries zero-in on their true reward. This not only enables us combine sources, but it also informs robot when should leverage each type information. Further, our accounts for human’s ability provide data: yielding user-friendly theoretically optimal. Our extensive simulated experiments studies Fetch mobile manipulator demonstrate superiority usability integrated framework.